Skip to content

Fix agent scan/get skipping apply for recorded patches (#454) - #456

Merged
Mikola Lysenko (mikolalysenko) merged 7 commits into
mainfrom
agent/fix-agent-apply-skipped-records
Oct 1, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 7 commits into
mainfrom
agent/fix-agent-apply-skipped-records

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #454

Summary

scan --mode agent, scan --sync, and agent-mode get now re-apply a patch that is already recorded in .socket/manifest.json. Before this change, a re-run after a reinstall (a fresh Hatch env, pip install --force-reinstall, a CI cache miss, a failed first apply, a --global-prefix reinstall) exited 0 with the package unpatched.

Root cause

Both agent-mode engines in crates/socket-patch-cli/src/commands/get.rs gated the nested apply on a manifest change:

  • download_and_apply_patches_with (get <purl|CVE|GHSA>, scan --mode agent, scan --sync, -g/--global-prefix): apply_lock and apply_failed both required downloaded > 0.
  • save_and_apply_patch (get <uuid>): the same gate, on changed.

A patch already recorded at the same uuid comes back skipped, so the installed tree was never checked.

Fix

The diff is 3 files: get.rs, one updated test, one new test file.

  • FetchBatch.already_recorded counts the manifest-store skipped patches whose same uuid is already recorded.
  • download_and_apply_patches_with runs the nested apply when downloaded + already_recorded > 0 (and not --save-only). apply_failed and applied use the same count, so applied now means "selected recorded patches confirmed applied after this run". The manifest is still written only when a record changed, and apply is a no-op on already-patched files, so an in-sync re-run changes nothing on disk.
  • save_and_apply_patch applies unless --save-only. The "nothing to update." suffix now prints only when no apply follows, decided by apply_lock.is_none().
  • --save-only keeps its record-only intent.

No wrapper (npm/, pypi/, gem/) changes are needed. They dispatch to the binary.

Test evidence

New crates/socket-patch-cli/tests/in_process_agent_reapply.rs (hermetic npm fixture, wiremock API). Each test records and applies once, reinstalls the pristine file, then re-runs. The "without fix" column comes from the same tests run against get.rs with the fix stashed:

Test Without fix With fix
agent_scan_reapplies_already_recorded_patch_after_reinstall FAIL (file unpatched) ok
scan_sync_reapplies_already_recorded_patch_after_reinstall FAIL ok
get_uuid_agent_reapplies_already_recorded_patch_after_reinstall FAIL ok
get_purl_agent_reapplies_already_recorded_patch_after_reinstall FAIL ok
agent_scan_rerun_fails_when_recorded_patch_cannot_be_applied (--strict, expects exit 1) FAIL (exit 0) ok
agent_scan_in_sync_rerun_is_a_clean_noop (control: manifest bytes unchanged) ok ok
get_save_only_of_recorded_patch_still_does_not_apply (control) ok ok

One existing test pinned the bug and is updated. scan/scan_sync_e2e.rs::scan_apply_with_existing_blob_uses_local_cache pre-staged a same-uuid record over a pristine install and asserted the file stays unpatched with applied: 0. It now asserts the record is still skipped, there's no download, and the manifest is untouched, while the cached blob gets applied (applied: 1, file holds the patched bytes).

Local checks:

  • cargo clippy --workspace --all-features -- -D warnings: clean.
  • rustfmt --check on the new test file: clean. main isn't rustfmt-clean and CI doesn't gate on fmt. A workspace-wide reformat that slipped into 175f3a4 was reverted in the head commit, so only the touched code changes.
  • Before the revert: cargo test -p socket-patch-cli --all-features --lib --bins plus 20 get/scan/apply integration binaries passed 1293, failed 0.
  • After the revert, on the final head: lib, scan, get, in_process_get, in_process_scan, covgap_commands_get, in_process_agent_reapply passed 1133, failed 0.
  • A full cargo test --workspace --all-features couldn't finish locally because the sandbox ran out of disk while linking about 200 test binaries. Before that, the only failures were 3 covgap_commands_vendor read-only-dir tests that need a non-root user (the sandbox runs as uid 0), in vendor code this PR doesn't touch.
  • CodeQL: the test helpers avoid "uuid" in their names. CodeQL treats that word as sensitive, so a helper named get_uuid_args made existing stderr lines look like new cleartext-logging findings.

CI on head 35ee888: all 391 non-skipped checks pass (6 skipped), including coverage, the full e2e matrix, and CodeQL ("No new alerts in code changed by this pull request"). Bugbot found no issues.

Per-issue checklist

Related, not fixed here: #424 (scan JSON drops the apply failure detail on the first run).

🤖 Generated with Claude Code


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
A re-run of scan --mode agent, scan --sync, get <uuid> or get <purl>
after the package was reinstalled left it unpatched while exiting 0.
These tests reinstall the pristine file between runs and expect the
recorded patch to be applied again (#454).

Assisted-by: Claude Code:claude-opus-5-5
scan --mode agent, scan --sync and agent-mode get only ran the nested
apply when they downloaded a new or updated record. When the patch was
already in .socket/manifest.json, the installed copy was never checked,
so after a reinstall (a fresh Hatch env, a CI cache miss, a failed
first apply) the run exited 0 with the package unpatched.

Run the nested apply for every selected patch that is recorded,
already-recorded ones included, unless --save-only. Apply is a no-op on
already-patched files, and a failing apply now fails the run.

Fixes #454

Assisted-by: Claude Code:claude-opus-5-5
Comment thread crates/socket-patch-cli/src/commands/get.rs Fixed
CodeQL traced the --save-only flag from GetArgs, which also carries
the API token, into the stderr save summary. Decide the "nothing to
update" suffix from whether an apply follows instead, which is the
same condition without passing argument data into the log line.

Assisted-by: Claude Code:claude-opus-5-5
scan_apply_with_existing_blob_uses_local_cache pinned the #454 bug: a
patch already recorded at the same uuid, over a pristine install, was
left unapplied. The record is still skipped (no download, manifest
untouched), but the cached blob is now applied to the install.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 1, 2026 10:51
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread crates/socket-patch-cli/src/commands/get.rs Fixed
CodeQL's name heuristics treat "uuid" as sensitive, so the new test
helper get_uuid_args() showed up as a fresh taint source for existing
stderr lines in get and scan. Rename the helpers; behavior unchanged.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

A workspace-wide cargo fmt run reformatted 125 files this fix does
not touch (main is not rustfmt-clean and CI does not gate on it).
Restore them to main so the PR only carries the fix and its tests.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 35ee888. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 1, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[burn-down agent] Ready for review on 35ee888: mergeable, 0 commits behind main.

  • CI: 399/405 check runs green, 6 skipped by workflow conditions, 0 failing.
  • Bugbot: reviewed 35ee888, no new issues. Both earlier get.rs threads are outdated and resolved.
  • For the reviewer: agent-mode apply now also runs when the patch is already in the manifest (already_recorded). Check that applied counting the recorded patches is the semantics you want.

Generated by Claude Code

@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit a36432e into main Oct 1, 2026
405 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-agent-apply-skipped-records branch October 1, 2026 16:51
Mikola Lysenko (mikolalysenko) added a commit that referenced this pull request Oct 5, 2026
…GELOG sync/fold, blocker gate) (#643)

* Add scripts/release.py: release-train stamp, versions, CHANGELOG, blockers

PR 1 of the weekly release train (docs/release-train/DESIGN.md §7). One
stdlib-only Python file with argparse subcommands:

- stamp <V> [--check]: offline, byte-deterministic version stamp of
  Cargo.toml (workspace version + =V core pin), Cargo.lock (source-less
  workspace-member entries, so --locked builds), the 15 npm manifests and
  npm/socket-patch/package-lock.json (JSON edit; platform entries for any
  other version are dropped instead of re-resolved over the network, which
  ends the #233/#235 lock-drift class).
- semver: X.Y.Z and X.Y.Z-rc.N only, semver precedence.
- next-version: from git tags + burned release/* branches + the
  [Unreleased] headings of sync-main(C) computed in memory, never from
  main's Cargo version. New majors need APPROVED_MAJORS (5 is
  pre-approved) and are never skipped; otherwise refused with an error.
- changelog cut|promote|sync-main|check: rc sections reach main on every
  rc; promotion folds rc.1..rc.K into one [X.Y.Z] section and returns
  later abandoned rc blocks to [Unreleased]. Exact-match, deterministic,
  idempotent; newer [Unreleased] entries are never touched.
- notes: release notes with a link to the open-P1 query, never titles.
- blockers --base <sha>: the §3.5 release-blocker rule over REST
  (injectable transport); any API error blocks.

Tests: scripts/tests/test_release.py with temp git repos driven through
the train timeline and recorded REST shapes under
scripts/tests/fixtures/release/.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Stamp versions offline and let release-lint accept rc versions

- scripts/version-sync.sh is now a thin wrapper over `release.py stamp`
  (same CLI contract; also stamps Cargo.lock's workspace entries). On the
  clean tree `version-sync.sh 4.0.0` is a byte no-op.
- scripts/release-lint.sh: the grammar accepts X.Y.Z and X.Y.Z-rc.N;
  check 2 is `release.py stamp --check` (byte compare, offline, no
  clean-tree requirement, writes nothing); check 3 is `release.py
  changelog check` (rc sections; a stable fails while rc sections remain
  unfolded); new --stable-only and --tag-exists.
- ci.yml release-readiness: the rolling `release-sync` PR (which moves
  main to the newest cut tag, rc or stable) runs `release-lint.sh
  --tag-exists`; every other PR keeps today's behavior.
- release.yml (legacy pipeline until the train replaces it): lint with
  --stable-only so it can never publish an rc as Latest/npm latest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Remove the version-bump workflow and bump-version.sh

version-bump.yml's unsigned push is rejected by the main ruleset and it
never ran; the release train cuts versions with scripts/release.py
instead. The CHANGELOG header and docs/releasing.md now point at
docs/release-train/DESIGN.md (the full runbook rewrite is PR 4), the
interim manual bump uses `release.py changelog cut` + version-sync.sh,
and ci.yml stops shellchecking the deleted script.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Add the release-train design with maintainer decisions D1-D4

D1: GitHub App socket-patch-release + refs/tags/v* ruleset (App-only
bypass); the App mints the tag in the publish job (PR 3). D2: version and
CHANGELOG reach main on every rc via the release-sync PR, folded at
promotion. D3: routines run as mikolalysenko (a routine actor, not an
approver) until a bot exists; npm stable is direct OIDC; newest-line
hotfixes only. D4: the first train release is 5.0.0 (pre-approved major).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Fix PR 1 review findings in release.py, release-lint and the CI gate

CHANGELOG (sync-main / cut / promote):
- The fold no longer classifies rc sections by rc number. Every rc
  section whose core shipped is replaced by its blocks (tag text) minus
  [S]'s blocks, as a multiset shared with the [Unreleased] removal. A
  train rc cut beside a same-core hotfix now returns its entries instead
  of losing them, whichever was cut first. TagSource.promoted_from is gone.
- Matching counts occurrences: a shipped block removes one occurrence,
  so a repeated entry ("- Updated dependencies.") survives an unmerged
  sync PR and no longer changes the bump level.
- Returned blocks go before the blocks already in their subsection, and
  a cut orders its ### subsections canonically (breaking, then Keep a
  Changelog, then the rest). The next cut is now byte-identical with or
  without the sync PR, including subsection order left by shipped history.
- CRLF CHANGELOGs round-trip, and every generated line uses CRLF.

Blocker gate:
- Fails closed unless GET /labels/release-blocker returns that exact
  name. Label names compare case-insensitively. labeled events since L
  are candidates, and a label that vanished without an unlabeled event
  still blocks. Deleted, converted and transferred issues block.
- RELEASE_APPROVERS / RELEASE_ROUTINE_ACTORS split on commas and
  whitespace and accept a leading @. A malformed login, an empty list, or
  no trusted approver blocks with a config error, in cmd_blockers and in
  evaluate_blockers.
- since = committer date of merge-base(L, base), clamped to the base
  date, not a forgeable tag date. A since later than the base fails
  closed.
- PR-merge closes (closed event with commit_id null, recorded from #454)
  resolve through the closing PR's merge_commit_sha being in base.
- Malformed event shapes fail closed.

Lint / CI:
- release-lint check 2 also runs the new offline `release.py
  npm-lock-check`: the wrapper lock's packages[""] must match
  package.json, and every non-optional dependency needs a node_modules
  entry. This restores the dependency-drift check the networked lock
  refresh used to give.
- ci.yml takes the release-sync --tag-exists path only for a same-repo
  release-sync branch into main.
- rel.Git ignores GIT_DIR/GIT_WORK_TREE and similar variables.

Tests:
- StampTests are hermetic: they run on a temp tree stamped to a fixed
  baseline (4.0.0, and 5.0.0-rc.1 via a subclass), so the suite passes on
  main after the release-sync PR.
- The temp repos ignore the git env and global config.
- New coverage: release-lint (plain and --tag-exists) on a sync-main'ed
  tree at an rc, the CI gate step run with stubs, and a regression test
  for each finding.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* DESIGN.md: align with the PR 1 review fixes

- §1: rc.2's section is synced to main and its unshipped blocks return
  to [Unreleased] when the stable is synced.
- §2: the release.py list adds semver, npm-lock-check, next-version and
  changelog.
- §3.1: U = [Unreleased] of sync-main(C) computed in memory (this
  replaces the pre-D2 "minus L's section" rule).
- §3.4: hotfix cuts use --no-sync, and same-core train rcs return to
  [Unreleased].
- §3.5: the label check, case-insensitive names, labeled-event
  candidates, vanished labels and gone issues, the merge-base `since`,
  strict config parsing, and PR-merge close resolution.
- §3.7: multiset fold, chronological returns, canonical cut order,
  CRLF, and the same-repo-only release-sync CI path.
- §4: npm-lock-check and the ci.yml condition.
- §5 I1 and PR 4: the stable tree is stamp + promote (fold), not a
  heading rename.
- S5: the release-blocker label must exist before the gate can pass.
- §7 PR 1: the accept list adds the new scenarios.
- §8: APPROVED_MAJORS is kept (D4), and multiset counting is kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Fix PR 1 re-review findings: multiset fold, since lookback, CRLF stamp

- promote keeps every occurrence when folding rcs, so [X.Y.Z] is the
  multiset sum of its rcs and sync-main charges them with the same count
  (a repeated entry no longer returns as unshipped or raises the bump);
  a later abandoned rc returns all of its blocks.
- sync-main charges the rc sections on main against [S] before removing
  the rest of [S] from [Unreleased], so a newer identical entry keeps its
  place whether or not the release-sync PR merged.
- blockers: since is never later than base - 35 days, so a forged high
  stable tag at the base cannot shrink the candidate window further; S9
  is a hard prerequisite for live gate runs.
- blockers: a PR-merge close resolves only through PRs merged by the
  close actor (merged_by.login, recorded for #456).
- stamp keeps each file's line endings (CRLF checkouts check clean).
- sync-main computes CHANGELOG and stamp before writing anything.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Fix release lint assertions under GitHub Actions

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

4 participants